Moving picture encoding system

ABSTRACT

A moving picture encoding system capable of bit rate control, by which moving pictures are encoded while maintaining high quality even when there are substantial changes in the size of objects and the characteristics of texture is provided. A predictive area calculating parameter extracting means obtains a predictive area calculating parameter to describe a function that indicates temporal variations in the area based on the history of the area data of an object. Besides, a bit number model parameter calculating means finds a bit number model parameter to describe a parameter for a bit number model used in modeling the generated bit number per unit area. A target bit number calculating means estimates a predictive value of the generated bit number for the uncoded VOPs based on the predictive area calculating parameter and the predictive bit number calculating parameter, and accordingly, allocates the remaining allocatable bits to decide a target bit number for the next VOP to be encoded. A coding means encodes the VOP according to the target bit number.

BACKGROUND OF THE INVENTION

The present invention relates to a moving picture encoding system, inwhich a moving picture is encoded with respect to each object, or amoving picture is divided into plural sections and bit allocation forencoding is decided with respect to each section, in particular, relatesto improvement in bit rate control.

DESCRIPTION OF THE RELATED ART

In encoding moving pictures, there is a system employed by Test Model 5(hereinafter referred to as TM-5) of MPEG-2 (Moving Picture ExpertsGroup, Phase 2) as a coding control system, in which the target bitnumber of each frame is determined based on the amount of availablebits, and the number of bits generated for the frame is controlled so asto approximate the target number (manuscript: “March, 1993, ISO/IEC JTC1/SC 29/WG 11/NO400”).

FIG. 1 is a block diagram showing a conventional bit rate controlscheme. The rate control scheme comprises a coding means 1001, a targetbit rate calculating means 1002 and a bit number model parametercalculating means 1006. The coding means 1001 is supplied with imagedata and outputs of the target bit rate calculating means 1002, andoutputs bit streams as the first output. Its second output (generatedbit number) is sent to the target bit rate calculating means 1002 andthe bit number model parameter calculating means 1006, and the thirdoutput (coding information) is sent to the bit number model parametercalculating means 1006. The bit number model parameter calculating means1006, where the second and third outputs of the coding means 1001 areinputted, sends its output (bit number model parameter) to the targetbit rate calculating means 1002. The target bit rate calculating means1002 is supplied with the second output of the coding means 1001, theoutput of the bit number model parameter calculating means 1006 andavailable bits information, and sends its output (target bit number) tothe coding means 1001.

In the coding means 1001, the inputted image data are encoded under thecoding control so that the number of bits used for each frame meets thetarget supplied from the target bit rate calculating means 1002.Subsequently, the number of generated bits and coding information ofeach frame are outputted along with encoding results outputted as bitstreams. Here, the coding information is quantization parameters used inencoding.

The bit number model parameter calculating means 1006 calculatesparameters for modeling the bit number. In the system of TM-5, thisparameter is a complexity index defined as a product of the generatedbit number and the quantization parameter. Namely, assuming that: X_(I),X_(P), X_(B); S_(I), S_(P), S_(B); and Q_(I), Q_(P), Q_(B) denote thecomplexity index, the number of generated bits, and the average value ofthe quantization parameter for each of I, P and B pictures,respectively, following expressions (1), (2) and (3) can be formed.$\begin{matrix}{S_{I} = \frac{X_{I}}{Q_{I}}} & (1) \\{S_{P} = \frac{X_{P}}{Q_{P}}} & (2) \\{S_{B} = \frac{X_{B}}{Q_{B}}} & (3)\end{matrix}$

Obtained complexity indexes X_(I), X_(P) and X_(B) are outputted to thetarget bit rate calculating means 1002 as model parameters.

The target bit rate calculating means 1002 estimates the target bitnumber for each picture according to allocatable bit number information,the model parameter and the number of generated bits, and outputs theresult to the coding means 1001. In MPEG-2, bit allocation is performedwith respect to each GOP (Group Of Picture). First, the number ofavailable bits for a GOP is found by the allocatable bit numberinformation. Subsequently, the number of bits which have been used forcoding the previous pictures in the GOP are deducted from the number ofavailable bits to estimate the number of bits that are available for theremainder of the GOP (hereinafter referred to as R). After that, usingpredetermined constants K_(P) and K_(B), which indicate the roughnessratio in quantization of P and B pictures, the target bit number for therespective I, P and B pictures (hereinafter referred to as T_(I), T_(P)and T_(B), respectively) is calculated by following expressions (4), (5)and (6). $\begin{matrix}{T_{I} = \frac{R}{1 + {\frac{X_{P}}{K_{P}X_{I}}N_{P}} + {\frac{X_{B}}{K_{B}X_{I}}N_{B}}}} & (4) \\{T_{P} = \frac{R}{N_{P} + {\frac{X_{B}K_{P}}{X_{P}K_{B}}N_{B}}}} & (5) \\{T_{B} = \frac{R}{N_{B} + {\frac{X_{P}K_{B}}{X_{B}K_{P}}N_{P}}}} & (6)\end{matrix}$In (4), (5) and (6), N_(P) and N_(B) denote the number of P and Bpictures that remain to be coded in the GOP. The target bit numbers areoutputted to the coding means 1001, and used for rate control.

The above-mentioned rate control scheme does not involve object shapeinformation since it is developed for MPEG-2. For MPEG-4, in whichobject-based encoding is performed, there are proposed schemes disclosedin Japanese Patent Application Laid-Open No. 2000-50254, and on page 186to 199 of the document “IEEE Transactions on Circuits and Systems forVideo Technology, Vol. CSVT-9, No. 1, February 1999”. In the schemes, asecond-order rate-distortion curve, which is described on page 246 to250 of “IEEE Transactions on Circuits and Systems for Video Technology,Vol. CSVT-7, No. 1, February 1997”, is employed. Namely, belowexpressions (7), (8) and (9) replace (1), (2) and (3) in modeling thenumber of bits. $\begin{matrix}{S_{I} = {\frac{X_{I}D}{Q_{I}} + \frac{Y_{I}D}{Q_{I}^{2}}}} & (7) \\{S_{P} = {\frac{X_{P}D}{Q_{P}} + \frac{Y_{P}D}{Q_{P}^{2}}}} & (8) \\{S_{B} = {\frac{X_{B}D}{Q_{B}} + \frac{X_{B}D}{Q_{B}^{2}}}} & (9)\end{matrix}$In (7) to (9), D denotes the mean absolute difference (MAD) of a motioncompensation predictive difference signal. Besides, X_(I), Y_(I), X_(P),Y_(P), X_(B), Y_(B) and D are the model parameters of bit quantity. Thebit number model parameter calculating means 1006 calculates the valuesby a least squares estimation based on the values of the quantizationparameters for past encoded frames, the MAD and the data on the numberof generated bits. The calculation proceeds as follows: first,substitute expressions (10) and (11) for (8) and (9) to eliminate Q_(P)and Q_(B), and then, assuming that S_(I)=T_(I), S_(P)=T_(P) andS_(B)=T_(B), substitute (7), (8) and (9) for (12) to achieve the valueof Q_(I). Thus, target bit numbers T_(I), T_(P) and T_(B) for I, P and Bframes are calculated.Q _(P) =K _(P) Q _(I)  (10)Q _(B) =K _(B) Q _(I)  (11)T _(I) +N _(P) T _(P) +N _(B) T _(B) =R  (12)

In Japanese Patent Application Laid-Open No. 2000-50254, there is alsodisclosed a scheme, in which the bits for a frame estimated as above areallocated among a plurality of objects included in the frame. Accordingto the scheme, the target bit number for a VOP (Video Object Plane) isdetermined by allocating bits for each object proportional to aweighting factor, which is the weighted average of the size, motion andactivity of the object.

First, bit allocation for each frame is decided by using expression(13). $\begin{matrix}{T = {\sum\limits_{j}\quad\left\{ {{\alpha\quad S_{j}} + {\left( {1 - \alpha} \right){\max\left( {L,\frac{R}{mN}} \right)}}} \right\}}} & (13)\end{matrix}$In (13), L indicates the number of bits required to assure a minimumquality. In addition, m denotes the number of objects, N denotes thenumber of frames which remain to be coded, S_(j) denotes the number ofbits generated for the j-th object of the previous frame, and a denotesa weighting factor. Here, a value of weighting factor α is 0.2. Morespecifically, remaining available bits R are allocated equally amongVOPs, and then the allocation of each VOP is adjusted according to thenumber of bits generated in the previous coding. Thus, the total targetbit number for objects included in a frame is estimated.

Next, the total target bit number T obtained by (13) is refined bybuffer processing. After that, the total target bits T are allocated foreach object according to the weight given by expression (14).w _(s)SIZE_(j) +w _(m) MOT _(j) +w _(v) MAD _(j) ²  (14)In (14), SIZE_(j), MOT_(j) and MAD_(j) indicate the size, motion vector,and MAD of the j-th object, respectively. Besides, w_(s), w_(m) andw_(v) are weighting factors with values of w_(s)=0.4, w_(m)=0.6 andw_(v)=0.0, or w_(s)=0.25, w_(m)=0.25 and w_(v)=0.5.

In the conventional rate control techniques, the bit rate is estimatedwith disregard to variations in the number of bits due to changing sizeof each object. In TM-5, this is not a problem since the size of apicture plane being subject to encoding stays unchanged. However, inobject-based coding such as MPEG-4, when there is a sudden increase inobject's size, bit allocation is not carried out successfully toward theend of a GOV (Group Of VOP) despite the substantial increase in theamount of generated bits. Consequently, picture quality is deteriorated,and encoding cannot be performed within the target bit number.

In the scheme disclosed in Japanese Patent Application Laid-Open No.200050254, the object size information is taken into account byexpression (14) on the occasion of allocating the total target bits fora frame among objects included in the frame. However, since theinformation is unconsidered at the point of determining the total targetby expression (13), the same problems occur.

Moreover, in expression (13) of the above scheme for estimating thetarget bit number for each frame, it is assumed that the VOP rate ofrespective objects is uniform. Therefore, when there is difference inthe frame rate among objects, the estimate of the target bit number forthe frame is irrelevant.

Furthermore, in the rate control for video objects according to theabove scheme, bits are allocated based on weight, which is given as thelinear weighted sum of the size (area), motion and MAD of an object. Theterm of the MAD, however, is independent of the size of the object, andmay give a large value even when the object is very small. Consequently,a large amount of bits are allocated for a small object when the valueof its MAD is large. As a result, the number of bits to be allocated forother objects is reduced on the whole, which causes deterioration in thepicture quality of the other objects.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a movingpicture encoding system capable of bit rate control, by which movingpictures are encoded while maintaining high quality even when there aresubstantial changes in the size of objects and the characteristics oftexture.

Besides, it is another object of the present invention to provide amoving picture encoding system capable of bit rate control, by which bitallocation is performed properly even when the frame rate varies witheach object.

In accordance with the first aspect of the present invention, there isprovided a moving picture encoding system for encoding moving picturesequences with respect to each object, comprising: a coding means forencoding object picture data consisting of time series sequences ofvideo object planes (VOPs), each of which is a picture image of at leastone object at a point of time, and shape information data indicating theshape of the object in each VOP while conducting bit rate control sothat the number of generated bits for each VOP meets a target bitnumber, and outputting coding information including a quantizationparameter used in encoding and the generated bit number along withobtained bit streams; an area calculating means for calculating the areaof the object in each VOP based on the shape information data, andoutputting the result as area data; a predictive area calculatingparameter extracting means for obtaining a function that indicatestemporal variations in the area of the object based on the history ofthe area data, and outputting a parameter specifying the function or apredictive value of the area obtained by the function as a predictivearea calculating parameter; a bit number model parameter calculatingmeans for calculating a parameter used in modeling the generated bitnumber per unit area of the object based on the coding information, thegenerated bit number and the area data, and outputting the result as abit number model parameter; a predictive bit number calculatingparameter extracting means for obtaining a function that indicatestemporal variations in the bit number model parameter based on thehistory of the bit number model parameter, and outputting a parameterspecifying the function or a predictive value of the bit number modelparameter obtained by the function as a predictive bit numbercalculating parameter; and a target bit number calculating means whichperforms a series of processes: calculating an uncoded VOP allocatablebit number that is the total number of allocatable bits for uncoded VOPsin a certain period of time based on allocatable bit number informationindicating the total number of allocatable bits for the VOPs in thecertain period of time and the number of generated bits for the encodedVOPs in the certain period of time, estimating the number of generatedbits for the uncoded VOPs based on the predictive area calculatingparameter and the predictive bit number calculating parameter,allocating the uncoded VOP allocatable bit number, calculating a targetbit number for the next VOP to be encoded, and outputting the target bitnumber; sequentially for each of VOPs in the certain period of time.

In accordance with the second aspect of the present invention, there isprovided a moving picture encoding system for encoding moving picturesequences with respect to each object, comprising: a storing means fortemporarily storing object picture data consisting of time seriessequences of video object planes (VOPs), each of which is a pictureimage of at least one object at a point of time, and shape informationdata indicating the shape of the object in each VOP; a coding meanswhich reads the object picture data and shape information data out ofthe storing means, encodes the data while conducting bit rate control sothat the number of generated bits for each VOP meets a target bitnumber, and outputs coding information including a quantizationparameter used in encoding and the generated bit number along withobtained bit streams; an area calculating means for calculating the areaof the object in each VOP based on the shape information data, andoutputting the result as area data; a bit number model parametercalculating means for calculating a parameter used in modeling thegenerated bit number per unit area of the object based on the codinginformation, the generated bit number and the area data, and outputtingthe result as a bit number model parameter; a predictive bit numbercalculating parameter extracting means for obtaining a function thatindicates temporal variations in the bit number model parameter based onthe history of the bit number model parameter, and outputting aparameter specifying the function or a predictive value of the bitnumber model parameter obtained by the function as a predictive bitnumber calculating parameter; and a target bit number calculating meanswhich performs a series of processes: calculating an uncoded VOPallocatable bit number that is the total number of allocatable bits foruncoded VOPs in a certain period of time based on allocatable bit numberinformation indicating the total number of allocatable bits for the VOPsin the certain period of time and the number of generated bits for theencoded VOPs in the certain period of time, estimating the number ofgenerated bits for the uncoded VOPs based on the area data and thepredictive bit number calculating parameter, allocating the uncoded VOPallocatable bit number, calculating a target bit number for the next VOPto be encoded, and outputting the target bit number; sequentially foreach of VOPs in the certain period of time.

In accordance with the third aspect of the present invention, there isprovided a moving picture encoding system for encoding each frame ofmoving picture sequences while conducting bit rate control with respectto each section of the frame, comprising: a coding means which issupplied with picture data, section information data indicating thesections in each frame of the picture data and a target bit number foreach section, encodes the data with respect to each section whileconducting bit rate control so that the number of generated bits foreach section meets the target bit number, and outputs coding informationincluding a quantization parameter used in encoding and the generatedbit number along with obtained bit streams; an area calculating meansfor calculating the area of the section in each frame based on thesection information data, and outputting the result as area data; apredictive area calculating parameter extracting means for obtaining afunction that indicates temporal variations in the area of the sectionbased on the history of the area data, and outputting a parameterspecifying the function or a predictive value of the area obtained bythe function as a predictive area calculating parameter) a bit numbermodel parameter calculating means for calculating a parameter used inmodeling the generated bit number per unit area of the section based onthe coding information, the generated bit number and the area data, andoutputting the result as a bit number model parameter; a predictive bitnumber calculating parameter extracting means for obtaining a functionthat indicates temporal variations in the bit number model parameterbased on the history of the bit number model parameter, and outputting aparameter specifying the function or a predictive value of the bitnumber model parameter obtained by the function as a predictive bitnumber calculating parameter; and a target bit number calculating meanswhich performs a series of processes: calculating an uncoded frameallocatable bit number that is the total number of allocatable bits foruncoded frames in a certain period of time based on allocatable bitnumber information indicating the total number of allocatable bits forthe frames in the certain period of time and the number of generatedbits for the encoded frames in the certain period of time, estimatingthe number of generated bits for each section in the uncoded framesbased on the predictive area calculating parameter and the predictivebit number calculating parameter, allocating the uncoded frameallocatable bit number, calculating a target bit number for each sectionin the next frame to be encoded, and outputting the target bit number;sequentially for each of frames in the certain period of time.

In accordance with the fourth aspect of the present invention, there isprovided a moving picture encoding system for encoding each frame ofmoving picture sequences while conducting bit rate control with respectto each section of the frame, comprising: a storing means fortemporarily storing picture data, section information data indicatingthe sections in each frame of the picture data; a coding means whichreads the picture data and section information data out of the storingmeans, encodes the data with respect to each section while conductingbit rate control so that the number of generated bits for each sectionmeets a target bit number for the section, and outputs codinginformation including a quantization parameter used in encoding and thegenerated bit number along with obtained bit streams; an areacalculating means for calculating the area of the section in each framebased on the section information data, and outputting the result as areadata; a bit number model parameter calculating means for calculating aparameter used in modeling the generated bit number per unit area of thesection based on the coding information, the generated bit number andthe area data, and outputting the result as a bit number modelparameter; a predictive bit number calculating parameter extractingmeans for obtaining a function that indicates temporal variations in thebit number model parameter based on the history of the bit number modelparameter, and outputting a parameter specifying the function or apredictive value of the bit number model parameter obtained by thefunction as a predictive bit number calculating parameter; and a targetbit number calculating means which performs a series of processes:calculating an uncoded frame allocatable bit number that is the totalnumber of allocatable bits for uncoded frames in a certain period oftime based on allocatable bit number information indicating the totalnumber of allocatable bits for the frames in the certain period of timeand the number of generated bits for the encoded frames in the certainperiod of time, estimating the number of generated bits for each sectionin the uncoded frames based on the area data and the predictive bitnumber calculating parameter, allocating the uncoded frame allocatablebit number, calculating a target bit number for each section in the nextframe to be encoded, and outputting the target bit number; sequentiallyfor each of frames in the certain period of time.

In accordance with the fifth aspect of the present invention, there isprovided a moving picture encoding method for encoding moving picturesequences with respect to each object, comprising the steps ofcalculating an uncoded VOP allocatable bit number that is the totalnumber of allocatable bits for uncoded VOPs in a certain period of timeby subtracting the number of generated bits for the encoded VOPs in thecertain period of time from the total number of allocatable bits for theVOPs in the certain period of time, estimating the number of generatedbits for all the uncoded VOPs, calculating a target bit number for thenext VOP to be encoded by allocating the uncoded VOP allocatable bitnumber, and encoding the VOP; sequentially for each of VOPs in thecertain period of time.

In accordance with the sixth aspect of the present invention, there isprovided a moving picture encoding method for encoding each frame ofmoving picture sequences while conducting bit rate control with respectto each section of the frame, comprising the steps of calculating anuncoded frame allocatable bit number that is the total number ofallocatable bits for uncoded frames in a certain period of time bysubtracting the number of generated bits for the encoded frames in thecertain period of time from the total number of allocatable bits for theframes in the certain period of time, estimating the number of generatedbits for all the sections in the uncoded frames, calculating a targetbit number for each section in the next frame to be encoded byallocating the uncoded frame allocatable bit number, and encoding theframe; sequentially for each of frames in the certain period of time.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and features of the present invention will become moreapparent from the consideration of the following detailed descriptiontaken in conjunction with the accompanying drawings in which:

FIG. 1 is a block diagram showing a conventional moving picture encodingsystem;

FIG. 2 is a block diagram showing a moving picture encoding systemaccording to an embodiment of the present invention; and

FIG. 3 is a block diagram showing a moving picture encoding systemaccording to another embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now to the drawings, a description of preferred embodiments ofthe present invention will be given in detail.

FIG. 2 is a block diagram showing a moving picture encoding systemaccording to an embodiment of the present invention.

The moving picture encoding system comprises: a coding means 101, atarget bit number calculating means 102, a predictive area calculatingparameter extracting means 103, a predictive bit number calculatingparameter extracting means 104, an area calculating means 105 and a bitnumber model parameter calculating means 106.

The coding means 101 is supplied with the output of the target bitnumber calculating means 102 (target bit number), shape information dataand object picture data, and generates bit streams as the first output.Its second output (generated bit number) is provided to the target bitnumber calculating means 102 and the bit number model parametercalculating means 106, and the third output (coding information) isprovided to the bit number model parameter calculating means 106.

The area calculating means 105 is supplied with the shape informationdata, and provides the predictive area calculating parameter extractingmeans 103 and the bit number model parameter calculating means 106 withits output (area data). The predictive area calculating parameterextracting means 103, which is supplied with the output of the areacalculating means 105, provides the target bit number calculating means102 with its output (predictive area calculating parameter).

The bit number model parameter calculating means 106, where the secondand third outputs of the coding means 101, and the output of the areacalculating means 105 are inputted, provides the predictive bit numbercalculating parameter extracting means 104 with its output (bit numbermodel parameter).

The predictive bit number calculating parameter extracting means 104 issupplied with the output of the bit number model parameter calculatingmeans 106, and provides the target bit number calculating means 102 withits output (predictive bit number calculating parameter). The target bitnumber calculating means 102, where allocatable bit number information,the second output of the coding means 101, and the outputs of thepredictive area calculating parameter extracting means 103 and thepredictive bit number calculating parameter extracting means 104 areinputted, provides the coding means 101 with its output (target bitnumber).

The shape information data indicate the shape of objects. A maskpicture, in which, for example, object regions are shown with 255 pixelsand the pixel value is set to 0 outside the object, corresponds to thedata. The object picture data indicate a sequence of picture images ofthe object extracted from each frame.

The shape information data and the object picture data are inputted tothe coding means 101, and encoded. The encoding of MPEG-4, for example,is applicable as a method of encoding. In this case, bit rate control isconducted so that the number of bits for each VOP (video object plane)meets a target bit number After that, bit streams obtained by theencoding are outputted together with the generated bit number and thecoding information.

The coding information is characteristic values found in encoding suchas activity or motion compensation predictive difference power, the meanabsolute difference (MAD) of a motion compensation predictive differencesignal, the quantization parameter, and motion vector information.

The shape information is also inputted to the area calculating means105. The area calculating means 105 calculates the area of an objectregion by counting pixels included in the object region according to theshape information. When the shape information is not binary butmultivalued, the shape information is binarized before calculating thearea. Alternatively, the area may be indicated by the number of blocks(macrobloks) including the object region instead of the number ofpixels. The calculated area data are inputted to the predictive areacalculating parameter extracting means 103 and the bit number modelparameter calculating means 106.

The predictive area calculating parameter extracting means 103 finds afunction approximate to the temporal variations of the object area basedon the history of the area data supplied from the area calculating means105. The predictive area calculating parameter extracting means 103 thenoutputs a parameter required to describe the function as a predictivearea calculating parameter.

For example, the object area A(t) at the time t (or VOP number) in a VOPis approximated by n-th order expression (15). $\begin{matrix}{{A(t)} = {\sum\limits_{j = 0}^{n}\quad{p_{j}t^{j}}}} & (15)\end{matrix}$In (15), parameter p_(j) (j=0, . . . , n) is the predictive areacalculating parameter. These values are calculated by the least squaresmethod etc. according to the values of the object area obtained inprevious VOPs. Besides, the predictive area calculating parameter may bethe actual predictive value of the object area in each VOP to be codedobtained by expression (15). In addition, it is also possible that theseveral types of functions are previously designated, and an index thatspecifies a function is outputted together with a parameter as thepredictive area calculating parameter.

In this case, a function, which most approximates variations of thearea, is found from among the designated types of functions, and aparameter thereof is obtained. Then, an index that indicates the type ofthe function and the parameter are outputted to the target bit numbercalculating means 102 as the predictive area calculating parameter.

On the other hand, the generated bit number and the coding informationoutputted from the coding means 101 are inputted to the bit number modelparameter calculating means 106. The operation of the bit number modelparameter calculating means 106 is basically the same as that of the bitnumber model parameter calculating means 1006, in which the relationshipbetween the generated bit number and the coding information is expressedin a mathematical model, and a parameter to describe the model iscalculated as a bit number model parameter. However, in the calculationof the bit number model parameter, the bit number model parametercalculating means 1006 models the number of bits for a whole pictureplane, while the bit number model parameter calculating means 106 modelsthe number of bits per unit area.

Consequently, the area data calculated by the area calculating means 105are also inputted to the bit number model parameter calculating means106 to be used in the calculation of the bit number model parameter.Incidentally, the model employed in the calculation is not restricted tothe model obtained by (1) to (3), or (7) to (9), but the other modelsmay be used.

Subsequently, the obtained model parameter is outputted to thepredictive bit number calculating parameter extracting means 104.

The predictive bit number calculating parameter extracting means 104finds a function approximate to the temporal variations of the modelparameter based on the history of the bit number model parametersupplied from the bit number model parameter calculating means 106. Thepredictive bit number calculating parameter extracting means 104 thenoutputs a parameter required to describe the function as a predictivebit number calculating parameter.

For example, the bit number model parameter X(t) at the time t (or VOPnumber) in a VOP is approximated by n-th order expression (16).$\begin{matrix}{{X(t)} = {\sum\limits_{j = 0}^{n}\quad{q_{j}t^{j}}}} & (16)\end{matrix}$In (16), parameter q_(j) (j=0, . . . , n) is the predictive bit numbercalculating parameter. These values are calculated by the least squaresmethod etc. according to the values of the bit number model parameterobtained in previous VOPs. Besides, the predictive bit numbercalculating parameter may be the actual predictive value of the bitnumber model parameter in each VOP to be coded obtained by (16). Inaddition, it is also possible that the several types of functions arepreviously designated, and an index that specifies a function isoutputted together with a parameter as the predictive bit numbercalculating parameter.

The predictive area calculating parameter outputted from the predictivearea calculating parameter extracting means 103 and the predictive bitnumber calculating parameter outputted from the predictive bit numbercalculating parameter extracting means 104 are inputted to the targetbit number calculating means 102. The target bit number calculatingmeans 102 is also supplied with the allocatable bit number informationand the generated bit number outputted from the coding means 101.

The target bit number calculating means 102 allocates allocatable bitsin a certain period of time for each VOP in the certain period of timeby using the predictive area calculating parameter and the predictivebit number calculating parameter, and thereby obtains the target bitnumber for each VOP. The number of allocated bits for each VOP iSoutputted as a target bit number from the target bit number calculatingmeans 102.

The certain period of time corresponds a time interval taken for a GOV.When the bit rate control is executed with respect to each GOV, first,the number of available bits for a GOV is calculated based on theallocatable bit number information. A coding bit rate specified by auser, for example, is used as the allocatable bit number information. Inthis case, the number of available bits for a GOV is calculated bydividing the number of allocatable bits according to the information bya VOP rate, and then multiplying the quotient by the number of VOPs inthe GOV.

Next, when there are VOPs, which have already been encoded, in the GOV,the number of bits used for encoding them is subtracted from the numberof available bits for the GOV to find remaining available bits.Subsequently, the target bit number for the next VOP to be coded iscalculated in consideration of the allocation of the available bits forremaining VOPs in the GOV.

In the following, the process of calculating target bit numbers T_(I),T_(P) and T_(B) for I, P and B VOPs will be described using the bitnumber models obtained by (1) to (3). Incidentally, while (1) to (3) areapplied to modeling the bit number for a whole picture plane in theabove-mentioned conventional scheme, in this embodiment, (1) to (3) areapplied to modeling the bit number per unit area.

Assuming that: the target bit number per unit area of I VOP is T, thetime for 1 VOP is t_(I), the area at time t is A(t), and the complexityindexes for I, P and B VOPs are X_(I)(t), X_(P)(t) and X_(B)(t),respectively; target bit numbers T_(I), T_(P) and T_(B) of the I, P andB VOPs at time t are expressed as follows: $\begin{matrix}{{T_{I}\left( t_{I} \right)} = {{TA}\left( t_{I} \right)}} & (17) \\{{T_{P}(t)} = {\frac{X_{P}(t)}{{X_{I}\left( t_{I} \right)}K_{P}}{{TA}(t)}}} & (18) \\{{T_{B}(t)} = {\frac{X_{B}(t)}{{X_{I}\left( t_{I} \right)}K_{B}}{{{TA}(t)}.}}} & (19)\end{matrix}$In (17), (18) and (19), K_(P) and K_(B) denote parameters to control theroughness of quantization for P and B pictures similarly to the systemof TM-5. Assuming that the number of remaining available bits is R,expression (20), which is a generalized expression of (12), can beformed. $\begin{matrix}{{{\sum\limits_{t \in \tau_{I}}\quad{T_{I}(t)}} + {\sum\limits_{t \in \tau_{P}}\quad{T_{P}(t)}} + {\sum\limits_{t \in \tau_{B}}\quad{T_{B}(t)}}} = R} & (20)\end{matrix}$In (20), τ_(I), τ_(P) and τ_(B) denote a set of time when the type ofVOP is 1, P and B, respectively, in the remaining VOPs included in aGOV. Incidentally, when there is no VOP of corresponding type in theremaining VOPs, each of τ_(I), τ_(P) and τ_(B) is a null set.

Thereby, the target bit number for 1 VOP at time t_(I) can be expressedby expression (21). Similarly, the target bit numbers for P VOP at timet_(P) and B VOP at time tB are calculated by following expression (22)and (23). $\begin{matrix}{{T_{I}\left( t_{I} \right)} = \frac{{X_{I}\left( t_{I} \right)}{A\left( t_{I} \right)}R}{\begin{matrix}{{{X_{I}\left( t_{I} \right)}{A\left( t_{I} \right)}} + {\frac{1}{K_{P}}{\sum\limits_{t \in \tau_{P}}\quad{{X_{P}(t)}{A(t)}}}} +} \\{\frac{1}{K_{B}}{\sum\limits_{t \in \tau_{B}}\quad{{X_{B}(t)}{A(t)}}}}\end{matrix}}} & (21) \\{{T_{P}\left( t_{P} \right)} = \frac{{X_{P}\left( t_{P} \right)}{A\left( t_{P} \right)}R}{{\sum\limits_{t \in \tau_{P}}\quad{{X_{P}(t)}{A(t)}}} + {\frac{K_{P}}{K_{B}}{\sum\limits_{t \in \tau_{B}}\quad{{X_{B}(t)}{A(t)}}}}}} & (22) \\{{T_{B}\left( t_{B} \right)} = {\frac{{X_{B}\left( t_{B} \right)}{A\left( t_{B} \right)}R}{{\sum\limits_{t \in \tau_{B}}\quad{{X_{B}(t)}{A(t)}}} + {\frac{K_{B}}{K_{P}}{\sum\limits_{t \in \tau_{P}}\quad{{X_{P}(t)}{A(t)}}}}}.}} & (23)\end{matrix}$

In the target bit number calculating means 102, the values of the areaand the complexity index concerning the remaining VOPs included in theGOV are calculated first using the predictive area calculating parameterand the predictive bit number calculating parameter. Then, the targetbit number is calculated by an expression selected from (21) to (23)according to the type (I, P or B) of the next VOP to be coded using theobtained values of the area and the complexity index.

Besides, the quadratic bit number models shown by (7) to (9) may also beemployed. In this case, functions that indicate the temporal variationsof bit number model parameters X_(I), Y_(I), X_(P), Y_(P), X_(B), Y_(B)and D are found, and parameters to describe the functions are thepredictive bit number calculating parameters. Incidentally, while (7) to(9) are applied to modeling the bit number for a whole picture plane inthe above-mentioned conventional scheme, in this embodiment, (7) to (9)are applied to modeling the bit number per unit area. Calculationprocess is the same as the above case.

The calculated target bit number is inputted to the coding means 101,and used for encoding the next VOP.

In the above description, the target bit number is determined based ononly the bit counts for texture, however, it may be determined inconsideration of bits for motion and shape.

In this case, the coding means 101 counts the number of generated bitsfor the motion, shape and texture individually, and outputs the resultsas the generated bit number.

The bit number model parameter calculating means 106 calculatesparameters for modeling the bit numbers for the motion and shape perunit area in addition to the bit number model parameter concerning thetexture. The bit number model parameter concerning the motion/shape issimply calculated by dividing the bit number for the motion/shape by thearea according to the area data supplied from the area calculating means105, and outputted.

The predictive bit number calculating parameter extracting means 104finds functions approximate to the temporal variations of the modelparameters based on the history of the bit number model parametersconcerning the motion and shape. The predictive bit number calculatingparameter extracting means 104 then outputs parameters required todescribe the functions as predictive bit number calculating parametersalong with the bit number model parameter concerning the texture.

The target bit number calculating means 102 calculates the predictiveamounts of the motion and shape bits per unit area for remaining VOPs byusing the predictive bit number calculating parameters concerning themotion and shape. Subsequently, each result is multiplied by thepredictive value of the area estimated according to the predictive areacalculating parameter outputted from the predictive area calculatingparameter extracting means 103, and thus the predictive numbers of themotion and shape bits are calculated. Then, the sum of the predictivevalues for the remaining VOPs is subtracted from the remainingallocatable bit number R, and the result is set anew as R in (21) to(23) to calculate the target bit number for the texture. Thus the bitrate control, in which the number of bits for the motion and shape arealso taken into account, can be performed.

While in the above description, each object is encoded under the bitrate control individually, it is possible that plural objects areprocessed at the same time, and then the target bit rates for VOPs ofeach object are decided.

Assuming that the number of objects is J, the shape information data andobject picture data supplied to the coding means shown in FIG. 2 are thedata of J objects. Likewise there are the area data, the predictive areacalculating parameter, the coding information, the generated bit number,the bit number model parameter, the predictive bit number calculatingparameter, the target bit number, and the bit streams for each of the Jobjects. The coding means 101, the area calculating means 105, thepredictive area calculating parameter extracting means 103, the bitnumber model parameter calculating means 106 and the predictive bitnumber calculating parameter extracting means 104 execute theabove-mentioned processes for each of the J objects.

On the other hand, the target bit number calculating means 102calculates the target bit number for the individual J objects based onthe allocatable bit number, and the predictive area calculatingparameter, the predictive bit number calculating parameter and thegenerated bit number for the individual J objects. In the following, thedescription will be given of the calculation procedure of the target bitnumber for each object in this case.

The quantization parameter for 1 VOP of the j-th object can be expressedas follows:Q _(I,j) =M _(j) Q  (24).In (24), Q_(I, j) denotes the quantization parameter for I VOP of thej-th object, and Q denotes the standard quantization parameter. Thevalue of Q varies depending on the allocatable bit number. Besides,M_(j) denotes the constant for adjusting the roughness of quantization.When M_(j) is set to 1 for all objects, all the objects are quantized inthe same roughness level. In other words, it is possible to makevariations in the roughness of quantization among objects by the value.For example, quantization can be controlled so that visually importantobjects are quantized finely and the others are quantized roughly. Inthe following, the process of calculating the target bit number for eachobject based on Q set to accord with the allocatable bit number will bedescribed.

The quantization parameters for P and B VOPs of the j-th object Q_(P, j)and Q_(B, j) are obtained by expressions (25) and (26) by using (24).Q _(P,j) =K _(P,j) Q _(I,j) =K _(P,j) M _(j) Q  (25)Q _(B,j) =K _(B,j) Q _(I,j) =K _(B,j) M _(j) Q  (26)In (25) and (26), K_(P, j) and K_(B, j) denotes the ratio of thequantization parameters for P and B VOPs to the quantization parameterfor I VOPs. Thereby the target bit numbers for I, P and B VOPs of thej-th object are defined by following expressions (27), (28) and (29),respectively. $\begin{matrix}{T_{I},_{j}{\left( t_{I} \right) = {{\frac{X_{I},_{j}(t)}{Q_{I},_{j}}{A_{j}(t)}} = {\frac{X_{I},_{j}(t)}{M_{j}Q}{A_{j}(t)}}}}} & (27) \\{T_{P},_{j}{(t) = {{\frac{X_{P},_{j}(t)}{Q_{P},_{j}}{A_{j}(t)}} = {\frac{X_{P},_{j}(t)}{K_{P},_{j}{M_{j}Q}}{A_{j}(t)}}}}} & (28) \\{T_{B},_{j}{(t) = {{\frac{X_{B},_{j}(t)}{Q_{B},_{j}}{A_{j}(t)}} = {\frac{X_{B},_{j}(t)}{K_{B},_{j}{M_{j}Q}}{{A_{j}(t)}.}}}}} & (29)\end{matrix}$

Assuming that, τ_(I, j), τ_(P, j) and τ_(B, j) denote a set of time whena VOP is I, P and B VOP types, respectively, in remaining VOPs includedin the GOV for the j-th object, expression (30) can be formed.$\begin{matrix}{{\sum\limits_{j}\quad\left( {{\sum\limits_{{t \in \tau_{I}},_{j}}\quad T_{I}},_{j}{(t) + {\sum\limits_{{t \in \tau_{p}},_{j}}\quad T_{P}}},_{j}{(t) + {\sum\limits_{{t \in \tau_{B}},_{j}}\quad T_{B}}},_{j}(t)} \right)} = R} & (30)\end{matrix}$Incidentally, when there is no VOP of the above types in the remainingVOPs, each of τ_(I), τ_(P) and τ_(B) is a null set. By substituting (27)to (29) for (30) and solving Q, the expression (31) can be achieved.$\begin{matrix}{Q = {\frac{1}{R}{\sum\limits_{j}\quad{\frac{1}{M_{j}}\begin{pmatrix}{{\sum\limits_{{t \in \tau_{I}},_{j}}\quad X_{I}},_{j}{{(t)A_{j}(t)} +}} \\{{\frac{1}{K_{P},_{j}}{\sum\limits_{{t \in \tau_{P}},_{j}}\quad X_{P}}},_{j}{{(t)A_{j}(t)} +}} \\{{\frac{1}{K_{B},_{j}}{\sum\limits_{{t \in \tau_{B}},_{j}}\quad X_{B}}},_{j}{(t){A_{j}(t)}}}\end{pmatrix}}}}} & (31)\end{matrix}$

Further, by substituting (31) for (27) to (29), the expressions (32),(33) and (34) are obtained $\begin{matrix}{{T_{I,k}\left( t_{I} \right)} = \frac{{X_{I,k}\left( t_{I} \right)}{A_{k}\left( t_{I} \right)}R}{M_{k}{\sum\limits_{j}\quad{\frac{1}{M_{j}}\begin{pmatrix}{{\sum\limits_{{t \in \tau_{I}},_{j}}\quad X_{I}},_{j}{{(t){A_{j}(t)}} +}} \\{{\frac{1}{K_{P},_{j}}{\sum\limits_{{t \in \tau_{P}},_{j}}\quad X_{P}}},_{j}{{(t)A_{j}(t)} +}} \\{{\frac{1}{K_{B},_{j}}{\sum\limits_{{t \in \tau_{B}},_{j}}\quad X_{B}}},_{j}{(t){A_{j}(t)}}}\end{pmatrix}}}}} & (32) \\{{T_{P,k}\left( t_{P} \right)} = \frac{{X_{P,_{k}}\left( t_{P} \right)}{A_{k}\left( t_{P} \right)}R}{K_{P,_{k}}M_{K}{\sum\limits_{j}\quad{\frac{1}{M_{j}}\begin{pmatrix}{{\sum\limits_{{t \in \tau_{I}},_{j}}\quad X_{I}},{{{j(t)}A_{j}(t)} +}} \\{{\frac{1}{K_{P},_{j}}{\sum\limits_{{t \in \tau_{P}},_{j}}\quad X_{P}}},_{j}{{(t)A_{j}(t)} +}} \\{{\frac{1}{K_{B},_{j}}{\sum\limits_{{t \in \tau_{B}},_{j}}\quad X_{B}}},_{j}{(t){A_{j}(t)}}}\end{pmatrix}}}}} & (33) \\{{T_{B,K}\left( t_{B} \right)} = \frac{{X_{B,_{k}}\left( t_{B} \right)}{A_{k}\left( t_{B} \right)}R}{K_{B,_{k}}M_{K}{\sum\limits_{j}\quad{\frac{1}{M_{j}}\begin{pmatrix}{{\sum\limits_{{t \in \tau_{I}},_{j}}\quad X_{I}},_{j}{{(t){A_{j}(t)}} +}} \\{{\frac{1}{K_{P},_{j}}{\sum\limits_{{t \in \tau_{P}},_{j}}\quad X_{P}}},_{j}{{(t)A_{j}(t)} +}} \\{{\frac{1}{K_{B},_{j}}{\sum\limits_{{t \in \tau_{B}},_{j}}\quad X_{B}}},_{j}{(t){A_{j}(t)}}}\end{pmatrix}}}}} & (34)\end{matrix}$Thus the target bit number for each object is calculated and outputtedto the coding means.

As is described above, even when there are plural objects in a frame,bit allocation can be performed properly. Besides, the target bitnumbers obtained by (32) to (34) are applicable to the case where theframe rate varies with each object.

Next, another embodiment of the present invention will be describedreferring to FIG. 3. This embodiment can be applied to the case wheredelay is admissible in encoding. Basically, the moving picture encodingsystem of this embodiment has the same configuration as that illustratedin FIG. 2 except that it does not include the predictive areacalculating parameter extracting means 103 and is provided with a buffer200.

In addition, a target bit number calculating means 202 and a bit numbermodel parameter calculating means 206 replace the target bit numbercalculating means 102 and the bit number model parameter calculatingmeans 106, respectively.

The buffer 200 is supplied with shape information data and objectpicture data, and its first output (shape information data) and thesecond output (object picture data) are sent to the coding means 101.

The area calculating means 105 is supplied with the shape informationdata before being inputted to the buffer 200, and its output (area data)is sent to the target bit number calculating means 202 and the bitnumber model parameter calculating means 206.

The target bit number calculating means 202 is provided with outputs ofthe predictive bit number calculating parameter extracting means 104 andthe area calculating means 105, allocatable bit number information, andthe second output of the coding means 101 (generated bit number), andits output is sent to the coding means 101. In other respects, themoving picture encoding system of this embodiment is the same as thesystem shown in FIG. 2.

The buffer 200 stores the inputted shape information data and the objectpicture data as long as a period of unitary time for the bit ratecontrol (for example, a span of a GOV). The coding means 101 of thisembodiment operates in the same manner as that in FIG. 2.

The operation of the bit number model parameter calculating means 206 isbasically the same as that of the bit number model parameter calculatingmeans 106 in FIG. 2, except that the bit number model parametercalculating means 206 stores the area data outputted from the areacalculating means 105 for a certain period of time, and calculates thebit number model parameter by using the area data of a VOP correspondingto the generated bit number outputted from the coding means 101.

The target bit number calculating means 202 calculates the target bitnumber in the same manner as the target bit number calculating means 102in FIG. 2. However, the target bit number calculating means 202 usesactual area data calculated previous to encoding by the area calculatingmeans 105 for the calculation of the target bit number, while the targetbit number calculating means 102 uses a predictive value of the areaA(t) for the calculation.

The operation of other parts is the same as that of moving pictureencoding means shown in FIG. 2.

In the moving picture encoding system illustrated in FIG. 3, the targetbit number calculating means 202 calculates the target bit number not bya predictive value of the area but by actual area data. Consequently,bit allocation can be performed with a higher degree of accuracycompared to the moving picture encoding system illustrated in FIG. 2.

While a description has been given of a system in which objects areencoded individually, the moving picture encoding system of the presentinvention is applicable to the bit allocation for encoding a frame thatis a combination of objects without making a separation into eachobject. In this case, a frame is divided into sections for each object,and the bit rate control is performed with respect to each section. Thatis, coding model parameters are calculated for each sectioncorresponding to an object, and thereby the target bit number for eachsection is calculated in the same manner as the above-mentionedcalculation of the target bit number for each object. Thus, each sectionis encoded under the individual bit rate control according to the targetbit number. On this occasion, it is not necessary to encode shapeinformation. The shape information is used for the purpose of the bitrate control only.

Consequently, even when the area of a visually important objectundergoes a lot of changes in a sequence, available bits can beallocated in a balanced manner. Besides, the moving picture encodingsystem of the present invention is also applicable to the case where apicture plane is divided into sections and the bit rate control isperformed for each section to encode a sequence of frames not includingplural separated objects. To take a screen of a picture phone as anexample, the region carrying a picture of a person and the other regionsare divided, and the target bit number is calculated likewise withrespect to each region. Thus, it is made possible to perform the bitrate control according to the importance of each region.

Additionally, the moving picture encoding system according to thepresent invention may be implemented by reading a program for executingits operation stored in a storing medium such as a CD-ROM, a floppy diskor a nonvolatile memory card by a computer.

While the preferred embodiments of the present invention have beendescribed using specific terms, such description is for illustrativepurposes only, and it is to be understood that changes and variationsmay be made without departing from the spirit or the scope of thefollowing claims.

1. A moving picture object coding method comprising the steps of:computing the area of an object based on the shape information of saidobject; predicting a bit number to be generated based on a change ofsaid area over time; and encoding the object using said predicted bitnumber.
 2. The moving picture object coding method as claimed in claim1, wherein said predicted bit number is prepared for a plurality offrames; and the allocation of bit numbers of every frame that areallocable to said plurality of frames is decided based on said predictedbit number.
 3. The moving picture object coding method as claimed inclaim 1, wherein said predicted bit number is prepared for a pluralityof objects; and the allocation of bit numbers to every object that areallocable to said plural of objects is decided based on said predictedbit number.
 4. The moving picture object coding method as claimed inclaim 1, wherein said change of said area over time is predicted.
 5. Amoving picture object coding apparatus comprising: means for computingthe area of an object based on shape information of said object; meansfor predicting a bit number to be generated based on a change of saidarea over time; and means for encoding said object using said predictedbit number.
 6. The moving picture object coding apparatus as claimed inclaim 5, wherein in computing said area of said object, said predictedbit number is prepared for a plurality of frames; and allocation of bitnumbers to every frame that are allocable to said plurality of objectsis decided based on said predicted bit number.
 7. The moving pictureobject coding apparatus as claimed in claim 5, wherein in computing saidarea of said object, said predicted bit number is prepared for aplurality of objects; and allocation of bit numbers to every object thatare allocable to said plurality of objects is decided based on saidpredicted bit number.
 8. The moving picture object apparatus as claimedin claim 5, wherein said change of said area over time is predicted.